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Abstract 

We propose a method to investigate modular structure in networks based on fitted probabilistic model, 
where the connection probability between nodes is related to a set of introduced local attributes. The 
attributes, as parameters of the empirical model, can be estimated by maximizing the likelihood function of 
the observed network. We demonstrate that the distribution of attributes provides an informative visulization 
of modular networks on low-dimensional space, and suggest the attribute space can be served as a better 
platform for further network analysis. 

PACS numbers: 89.75.HC, 89.20.Fb 
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Networks are widely used to model complex systems with many interactive units Qjl [2Q . Usually, 
each node in such a network represents a distinct individual, and a link is established based on 
certain measurement of interaction between particular pair of nodes. On the microscopic level, the 
underlying system is fully described by state of each unit defined by several local properties, which 
also determines the interactions among them through complicated coupling. Thus, the network 
would be completely determined if local properties of nodes and interactive functions were known. 
In many applications however, the representative network is the only available data. It is the 
purpose of the researchers to infer these crutial information on the underlying system through 
network analysis. For instance, if the interaction between two units mainly depends on their local 
properties, then it is reasonable to expect that the units which have similar connection patterns in 
network representation, will share some common features in their local properties, and therefore 
may have similar functioning in the underlying system. Although very friutful, the applicability 
of this type of analysis is limited by the gap existing between the network-level description and 
the underlying system. This is due to the fact that in most situations, the representitive network 
only models relationships or interactions among units, but not the associated properties or states 
of units which determine interactions. So for example, a minor change on the observed network 
may not be caused necessarily by small perturbations on local properties of units in the underlying 
system, since interactions may depend on local states in a highly nonlinear way. Similarly, the 
evolution of the unerlying system due to continuous change of local states may result in abrupt 
changes on the network structure, such as group merge or split. 

To better deal with these problems, a necessary step is to bridge these two deffierent descrip- 
tion levels. However it is generally impossible to completely reconstruct the intrinsic properties 
of nodes based solely on the strucure of a given network, since the mechanism determining a 
link may be very complicated. In this paper, we develope a method to describe the system in a 
middle level between the representitive network and the microscopic description through some 
simplifications. We regard the representitive network as one particular realization of an emprical 
probabilistic model. In this model, each pair of nodes has certain probability to be connected, and 
the connection probabilities depends on the distance between local attributes of the corresponding 
nodes through some function. The attributes, act as model parameters, can be estimated according 
to some statistical criterion to best interpret the observed connections. Since each data point in 
the space of attributes is associated with distinct node, their configuration provides an alternative 
representation of the observed network. 
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The advantages of introducing this new representation are twofolds. On the one hand, the con- 
figuration of the attributes provides an informative projection of the network on a low-dimensional 
space. The established attribute space can be taken as new platform for further network analysis, 
where the attribute vectors which have no difference with conventional data source, allow many 
well developed clustering techniques to be applied directly. On the other hand, the introduced 
local attributes are closely related to the unknown properties of the unerlying system by the com- 
mon observed network. It may be easier to model evolution of the underlying system as changes 
of attributes, and study the final influences on network structure through the empiral probability 
model. 

In this paper, we mainly focus on studying modular strucutres in networks. After describing 
technique details of modeling approach, we apply the proposed method to some artificial and 
real-world networks to demonstrate its usefulness on network visualzing and structure analysis. 
We also discuss a possible way to extend the proposed technique to deal with more complicated 
multi-layered modular network. 

Assume there is an imaginary probabilistic system with N nodes, which is characterized by 
a connection probability matrix C(N x N), where Cy represents the probability of connection 
between node % and j. The given network described by adjancency matrix A(N x N) is regarded 
as a realization of this imaginary system. In particular, Aij is treated as a random variable and 
its observed value is determined by a Bernoulli trial according to the probability c^-. For each 
node in the imaginary system, there associates a set of local attributes denoted by w k (l x m), k = 
1, 2, • • • , N, where the dimension m <C N '. The connection probability q,- is then assumed to 
depend only on the local attributes Wi and Wj, and can be written as q-, = f(w i: Wj). Obviously, 
this is a great simplication, and should be regarded as first order approximation. The function 
/ tying connection probability c^- and local attributes Wi and Wj should be chosen depending on 
the problem at hand. In our study, we are mainly interested in undirectional unweighted modular 
networks. Considering the symmetric constraint, we choose a function / which depends only on 
the distance Aw —\\ Wi — Wj ||. /(Aw) will be closed to 1 when Aw approaching to zeros, and 
decays rapidly with increasing Aw . A natrual choise of / is in Guassian form: f(w i) Wj) = 
g-iwi-wj^fijiwi-wj) ^ w h ere defined as 



is used as a rescaling factor to compensate the distortion when projecting the high dimensional net- 
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work into a low-dimensional Euclidian space due to the inhomogeneity of the degree distribution 
of the network. The connection probability thus becomes c^- = e~( Wi ~ Wj ^ T ' yij ( Wi ~ Wj \ Therefore, 
we have 

Pij = PiAij = 1) = e -K-^') T ^K-^) (1) 
Pij = P(A ij = 0) = l-P(A ij = l). 
Under this framework, the logarithmic likelihood function of observing particular network A is 

L = \nPr{A\w) = £ A l3 hi Pij + ^(1 - A tj ) ln(l - Pij ), (2) 

and 

^"/^ — Tx \x = ~L^ — Tx -^Ijkiwk-Wj). (3) 

dw k ^ p jfc (l - p jfc ) ^ p jk {l - p jk ) 

Naturally, attributes w k which maximize above likelihood function are desired. These maximum 
likelihood estimates can be obtained by minimizing — L following steps below: 

1. starting from arbitary initials w , 

2. caculate the derivatives dL/dw, 

3. search in this direction to get rj, so that w\ = w — gives smallest —L, 

4. set w = Wi, and go back to step 2. 

5. quit if stopping conditions are met. 

It should be noded that the explicitly casted functional relationship of attributes and connection 
probabilities may not be the true undelrying mechanism. This approach nevertheless, can be ap- 
plied as a useful technique to explore the desired structure in the network, providing the fitted 
model is good. 

To demonstrate how the distribution of attributes reflects the network topological structure, we 
first consider two simple networks: a line network and a circle network as shown in figure 1(a) 
and 1(b). In both networks, one prominent topological feature is the order of the nodes. The 
essential difference between these two networks lies on the different connection patterns of their 
ending nodes. Starting from independently random initial distribution (2D Gaussian distribution 
centered at origin and variance is 4), the attributes are finally settled down to certain specific 
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FIG. 1: (a) and (b) describe the line and circle networks, (c) and (d) show one typical configuration of 
attributes corresponding to network in (a) and (b) 

configuration after optimization procedure described above. One typical results are shown in figure 
1(c) and 1(d) which clearly mirror the essential structures of both networks. The corresponding 
nodes are ordered accordingly, and the ending nodes are correctly arranged to capture the different 
topological features. The apparent kink(s) observed are caused by the suboptimal (local optimal) 
nature of the solution generated by gradient based optimization procedure we adopted. In fact, 
better results (no obvious kinks) can be found seldomly if different initial conditions are tried but 
not guaranteed. The important point is that global optimal solution may not be necessary to capture 
the essential topological features. Moreover, if the dimension of the attributes is increased, e.g. 
form 2D to 3D, the kinks will disappear. However, by this way the model becomes more complex, 
and its visualization capability is decreased. 

Modular netowrks are prevalent in various fields. In some applications, the clustering struc- 
ture is the only one we are interested in. The proposed probabilistic modeling approach can be 
applied to modular networks successfully. Date points on attribute space generated by the method 
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are usually grouped corresponding to the intrinsic clustering structure, and provides a good vi- 
sual representation of the investigated network. As it said that a picture is more than thousands 
of words, in many situations, the picture in attribute space makes it relative easy to get a good 
guess about the number of clusters and the corresponding partitions even without further analysis. 
Figure 2 shows typical simulation results when the method is applied to an artificial clustered 




(a) (b) 

FIG. 2: (a)Diagram to show the structure of the modular network. The numbers in each circle indicate 
connection probabilities of nodes in same cluster. The numbers between circles indicate connection proba- 
bilities of nodes from different clusters. Three clusters are (nodes 1 to 25), (nodes 26 to 40) and (nodes 41 
to 60). (b)Attributes configuration on attribute space. Different symbols indicate different clusters. 

networks, which generated from a probabilistic model as described in figure 2(a). The final at- 
tributes configuration is shown in figure 2(b), where 3 clusters are disclosed clearly. The details 
of fitted model is shown in figure 3 to compare with the true underlying model. The similarity is 
striking. More careful investigation reveals that the fitting errors are mainly within the clusters. 
It is more likely that the connection probabilities determined by the fitted model deviate from the 
true value if the corresponding nodes are in the same cluster than they are in different clusters. 
Regarding structure analysis, this is actually more desired one comparing to homogeneous fitting 
errors because connections between nodes in different clusters usually convey much more infor- 
mation on structure than those in the same cluster. The main reason for the unsymmetric fitting 
errors is that the Gaussian function / adopted in our study is not matched with the uniform one 
in the underlying model. This observation may imply good generalization of proposed modeling 
approach since exact interactive function is not necessary to get useful results. 

The attribute space provides an alternative starting point to make clustering analysis of net- 
works. Since the attribute vectors have no difference with conventional data, there are a lot of 
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FIG. 3: Comparison of fitted model and true underlying model. Upper part shows the adjacency matrix 
of observed network (left) and the true underlying probabilistic model which is assumed to be unknown 
(right). Lower part shows the adjacency matrix of one surrogate network generated by the fitted model 
(left) and the fitted probabilistic model itself (right). 

well developed clustering techniques can be applied. Furthermore, the informative configuration 
of attribute vectors not only hint the cluster number, but also provide a good initial partition. With 
these two key parameters at hand, most clustering algorithms will converge quickly. We apply 
conventional K-means algorithm ||7|] to above network with initial partitions suggested by the at- 
tributes distribution. The algorithm always generate correct partition, and converge quickly after 
1 to 4 iteratives. 

For real world data, the network structure is more complicated. We apply the proposed ap- 
proach to several extensively studied real networks, including the karate club network[3], Ameri- 
can college football teams networkflg] and dolphins social network Q4£| . According to their degree 
distributions, the rescaling factor 7^ is set to be 1 for football team network, and 1/didj for other 
two networks. The initial attributes distributions are generated by 2D Gaussian distribution cen- 
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tered at origin and the variance is taken to be 4. The results for Karate club network are shown in 
figure 4. It can be seen clearly from figure 4(b) that the distribution of the attributes well reflect 
the modular structure (shown in figure 4(a)). More details of the fitted model are shown in figure 
4(c). One may find that the surrogate network generated by the fitted model is very closed to the 
observed one. 
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FIG. 4: (a) Karate club network, (b) Distribution of attributes, (c) Fitted probabilistic model of Karate clube 
network. Upper left: adjacency matrix of Karate club network. Upper right: decay of fitting error when 
training. Lower left: adjacency matrix of a surrogate network generated by fitted model. Lower right: the 
fitted probabilistic model. 

American college football teams network has many clusters involved. The fitted model never- 
theless successfully capture this complicated structure. The amazing representation of the network 
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FIG. 5: (a) (colored) Distribution of attributes of American college football teams network, (b) Fitted 
probabilistic model of American college football teams network. Upper left: adjacency matrix of football 
teams network. Upper right: decay of fitting error when training. Lower left: adjacency matrix of a 
surrogate network generated by fitted model. Lower right: the fitted probabilistic model. 

in atribute space (shown in figure 5(a)) not only manifests clustering structures, but also suggests 
that several wandering points may not be clearly classfied. It turns out that these nodes belongs 
to specific group (IA Independents) and can not be classifed as one cluster consistently. Fur- 
thermore, the particular node in the group of Conference USA which is classifed as member of 
the group of Western Athletic is not due to the flaw of the method but caused by the network 
construction! (]] . The attributes representation also suggests some groups may be further divided 
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FIG. 6: (a) Dolphins social network, (b) Distribution of attributes, (c) Fitted probabilistic model of Karate 
clube network. Upper left: adjacency matrix of dolphins network. Upper right: decay of fitting error when 
training. Lower left: adjacency matrix of a surrogate network generated by fitted model. Lower right: the 
fitted probabilistic model. 

into smaller subgroups such as the Mid American group. This observation is well supported by the 
connection patterns of nodes in this group. These useful information can not be easily obtained by 
conventional network clustering algorithms, and show the advantages of the proposed modeling 
approach. 

The dolphins social network is another widely studied example. It can be divided into two big 
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clusters. One of the cluster may be further divided into three small group as studied in [5]. The 
analysis results for dolphins social network are shown in figure 6. The distribution of the attributes 
correctly reflects 2 large clusters (as shown in figure 6(b)), but also indicates that there may exists 
2 or 3 smaller clusters in the left cluster. The interesting observation is that the attributes of the 
nodes corresponding to three small clusters suggested in ||5|] are indeed well grouped. Also, the 
surrogate network generated by the fitted model (as shown in figure 6(c)) shows striking similarity 
with the observed network. 

Decaying behaviors of fitting errors when the optimization procedure iterated unveil some com- 
mon feature, which can be seen from the subplots (upper right ones) in all examples studied as 
shown in figure 4(c), figure 5(c) and figure 6(c). It can be divided into two segments, a rapid de- 
crease phase and a gradual change phase. After closely monitoring the movement of the points in 
attribute space step by step, we find that the points are always firstly arranged according to globe 
structure of the network, and then fine adjusted within each cluster to generate better configura- 
tion. This may partially explained why the suboptimal solutions due to gradient based algorithm 
also give out good configuration in general. 

Although modular structures are most commonly observed and studied, there are many dif- 
ferent other structures. In our study, we choose Gaussian function to relate local attributes to 
connection probability, which is particular useful to capture clustering structure. The proposed 
modeling approach nevertheless is flexible enough to deal with other structures providing they can 
be well defined. For example, for a bipartite network, the function 1 — /, where / is similar to 
what used in the paper, may be a better choise. 

The proposed modeling approach can be easily extended to deal with more complicated group- 
ing structure than simple modular structure. Let us consider a simple exention of overlapping two 
clustering structure. For example, consider a group of students who make friends based on differ- 
ent factors such as personality or avocation. Now even if each friendship network based on any 
single factor were well clustered, the observed overall network may have structure quite different 
from the simple modular network. We call this kind of network multi-layered modular network. 
One such example of 2-layered network is shown in figure 7(a). This particular network is gener- 
ated by the following way. Suppose the nodes are numbered from 1 to 60. We first generate two 
modular networks separately. The first modular network has two clusters, one containing node 1 
to node 30 and the other containing node 31 to node 60. The connection probabilities are 0.7 for 
nodes in same cluster and 0.02 in different clusters. The second modular network also has two 
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clusters, one containing node 21 to node 40, and the other containing nodes 1 to 20 and nodes 41 
to 60. The connection probabilities are slightly different, which are 0.6 for nodes in same cluster 
and 0.01 for nodes in different clusters. The final network is generated by stacking two modular 
networks together, and removing repeated links. To explore such multi-layered structure by the 
proposed modeling approach, the most straight way is to extend the dimension of the attributes 
vector. Let w = [w^ T , w^ T ] T the new extended attributes. For each layer, the connection prob- 
ability can be written as p^ ~ /(|| — wf^ ||) and pf) ~ /(|| vJf* — wf^ ||). The connection 
probability of the observed network will be p^ = p^ + p\f — p\fp[f • Following the same 
procedure described above, we get the attributes representation in extended attribute space. The 
distribution of the attributes are shown in figure 7(b). Interestingly, the hidden clustering structure 
are unveiled in different subspace of the extended attribute space. 

In summary, we proposed a probabilistic modeling approach to analyze networks. Under this 
framework, the observed network can be regarded as a measurement of certain probabilistic sys- 
tem, where the connection probability of any pair of nodes depends on the properly rescaled dis- 
tance between the introduced local attributes of the corresponding nodes. It is remarkable that the 
configuration of the optimally estimated attributes well represents the intrinsic structure of the ob- 
served network, thus provides an very informative way to visualize networks in low-dimensional 
space. It can be more effective to make further network structure analysis based on the attribute 
vectors instead of observed network directly. The modeling approach can be easily extended to 
deal with more complicated structures such as multi-layered clustered network. 
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FIG. 7: (a) Multi-layered clustering netowrk. Upper left: adjancency matrix of observed network. Upper 
right: true unerlying generative model. Lower left: adjancency matrix of surrogate network. Lower right: 
fitted probabilistic model, (b) Distribution of attributes for 2-layered clustered network. Left: w^(l) vs. 
w W(2). Right: w^(l) vs. w( >(2). In both figures, different colors and different symbols are used to 
indicate which groups in different layer the nodes belong to. E.g., nodes in blue color (no.31 to 60) are in 
one cluster and nodes in red color (no.l to 30) are in another cluster at first layer. Similarly, the dotted nodes 
(no. 10 to 20 and no.41 to 60) and circled nodes (no. 21 to 40) are in different clusters at second layer. 
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